Computers accept human input in various ways. One of the most common input devices is the keyboard. Additional types of input mechanisms include mice and other pointing devices. Although useful for many purposes, keyboards and mice (as well as other pointing devices) sometimes lack flexibility. For these and other reasons, various alternative forms of input have been (and continue to be) developed. For example, electronic tablets or other types of electronic writing devices permit a user to provide input in a manner very similar to conventional writing. These devices typically include a stylus with which a user can write upon a display screen. A digitizer nested within the display converts movement of the stylus across the display into an “electronic ink” representation of the user's writing. The electronic ink is stored as coordinate values for a collection of points along the hne(s) drawn by the user. Speech is another alternative input form. Typically, the user speaks into a microphone, and the user's speech is digitized and recorded.
Before the information conveyed by speech or ink can be usefully manipulated by a computer, the speech or ink must usually undergo recognition processing. In other words, the graphical forms (ink) or sounds (speech) created by the user are analyzed by various algorithms to determine what characters (e.g., letters, numbers, symbols, etc.) or words the user intended to convey. Typically, the ink or speech is converted to Unicode, ASCII or other code values for what the user has recorded.
Various systems have been developed to recognize handwriting and speech input. In many cases, recognition algorithms involve isolating individual features of ink or speech input. These features are then compared to previously-generated prototype features. In particular, numerous samples of ink or speech are initially collected and used to create a database of prototype features against which an unknown ink or speech sample is compared. Based on the degree of similarity (or dissimilarity) between an input sample and one or more prototype features, one or more recognition results are generated for a particular user input.
Accuracy, or the closeness of a recognition result to what the user intended to convey, is an important criterion by which recognition systems are evaluated. For obvious reasons, a recognition engine must be reasonably accurate in order to be useful. However, “accuracy” is not easily defined or quantified. For example, a recognizer may work quite well in circumstance A, but not work well in circumstance B. Depending on what circumstances A and B are, this may or may not be significant. If a recognizer works well in circumstances that are very important to the end user, and only works poorly in circumstances that are of little consequence, the overall accuracy of the recognizer might be considered good. Conversely, a recognizer that provides highly accurate results in obscure circumstances but does not work well in more important circumstances might be considered inaccurate overall.
Accordingly, there remains a need for systems and methods of modeling the accuracy of signal processing engines used for, e.g., recognition of speech, handwriting and other complex forms of input.